{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ONNX ResNet Model\n",
    "\n",
    "This example will show inference over an exported [ONNX](https://github.com/onnx/onnx) ResNet model using Seldon Core. We will use the Seldon S2I wrapper for Intel's NGraph. The example follows this [NGraph tutorial](https://ai.intel.com/adaptable-deep-learning-solutions-with-ngraph-compiler-and-onnx/).\n",
    "\n",
    " Preequisites:\n",
    "   * torch==0.4.0\n",
    "   * torchvision==0.2.1\n",
    "   * protoc > 3.4.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "from torch.autograd import Variable\n",
    "import torch.onnx\n",
    "import torchvision\n",
    "from keras.applications.imagenet_utils import decode_predictions\n",
    "from keras.applications.resnet50 import preprocess_input\n",
    "from keras.preprocessing import image\n",
    "import ngraph as ng\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download ResNet pretrained model from Torch and export to ONNX file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ImageNet input has 3 channels and 224x224 resolution\n",
    "imagenet_input = Variable(torch.randn(1, 3, 224, 224))\n",
    "\n",
    "# Download ResNet (or construct your model)\n",
    "model = torchvision.models.resnet50(pretrained=True)\n",
    "\n",
    "# Export model to an ONNX file\n",
    "torch.onnx.export(model, imagenet_input, 'resnet.onnx')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test Model\n",
    "Load ONNX model into ngraph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ngraph_onnx.onnx_importer.importer import import_onnx_file\n",
    "\n",
    "# Import the ONNX file\n",
    "models = import_onnx_file('resnet.onnx')\n",
    "\n",
    "# Create an nGraph runtime environment\n",
    "runtime = ng.runtime(backend_name='CPU')\n",
    "\n",
    "# Select the first model and compile it to a callable function\n",
    "model = models[0]\n",
    "resnet = runtime.computation(model['output'], *model['inputs'])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test on an image of a Zebra."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "img = image.load_img('zebra.jpg', target_size=(224, 224))\n",
    "img = image.img_to_array(img)\n",
    "plt.imshow(img / 255.)\n",
    "x = np.expand_dims(img.copy(), axis=0)\n",
    "x = preprocess_input(x,mode='torch')\n",
    "x = x.transpose(0,3,1,2)\n",
    "preds = resnet(x)\n",
    "decode_predictions(preds, top=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Package Model Using S2I"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!s2i build . seldonio/seldon-core-s2i-python3-ngraph-onnx:0.1 onnx-resnet:0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker run --name \"onnx_resnet_predictor\" -d --rm -p 5000:5000 onnx-resnet:0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cd ../../../wrappers/testing && make build_protos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python ../../../wrappers/testing/tester.py contract.json 0.0.0.0 5000 -p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker rm onnx_resnet_predictor --force"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test using Minikube\n",
    "\n",
    "**Due to a [minikube/s2i issue](https://github.com/SeldonIO/seldon-core/issues/253) you will need Minikube version 0.25.2**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!minikube start --memory 4096 --feature-gates=CustomResourceValidation=true --extra-config=apiserver.Authorization.Mode=RBAC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!helm init"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!kubectl rollout status deploy/tiller-deploy -n kube-system"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!helm install ../../../helm-charts/seldon-core-crd --name seldon-core-crd  --set usage_metrics.enabled=true\n",
    "!helm install ../../../helm-charts/seldon-core --name seldon-core"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!kubectl rollout status deploy/seldon-core-seldon-cluster-manager\n",
    "!kubectl rollout status deploy/seldon-core-seldon-apiserver"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-python3-ngraph-onnx:0.1 onnx-resnet:0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!kubectl create -f onnx_resnet_deployment.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wait until deployment is running"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!kubectl get seldondeployments seldon-deployment-example -o jsonpath='{.status}'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cd ../../../util/api_tester && make build_protos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python ../../../util/api_tester/api-tester.py contract.json \\\n",
    "    `minikube ip` `kubectl get svc -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].spec.ports[0].nodePort}'` \\\n",
    "    --oauth-key oauth-key --oauth-secret oauth-secret -p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!minikube delete\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
